在工业应用中，无监督的异常检测是一项艰巨的任务，因为收集足够的异常样品是不切实际的。在本文中，通过共同探索锻造异常样品的有效生成方法和正常样品特征作为分割异常检测的指导信息，提出了一种新颖的自我监督指导性分割框架（SGSF）。具体而言，为确保生成的锻造异常样品有利于模型训练，提出了显着性增强模块（SAM）。 Sam引入了显着图来产生显着性Perlin噪声图，并制定了一种自适应分割策略，以在显着区域产生不规则的掩模。然后，将口罩用于生成伪造的异常样品作为训练的负样本。不幸的是，锻造和真实异常样品之间的分布差距使得基于锻造样品训练的模型难以有效定位真实异常。为此，提出了自我监督的指导网络（SGN）。它利用自我监督的模块提取无噪声的功能，并包含正常的语义信息作为分割模块的先验知识。分割模块具有正常模式段的知识，这些片段与指导特征不同。为了评估SGSF对异常检测的有效性，在三个异常检测数据集上进行了广泛的实验。实验结果表明，SGSF达到了最新的异常检测结果。

冻结预训练的主链已成为标准范式，以避免在几次分段中过度拟合。在本文中，我们重新考虑范式并探索一个新的制度：{\ em对骨干中的一小部分参数}进行微调。我们提出了一种解决过度拟合问题的解决方案，从而使学习新颖班级的模型概括更好。我们的方法通过奇异值分解（SVD）将主链参数分解为三个连续的矩阵，然后{\ em仅微调单数值}并保持其他冻结。上面的设计使模型可以在新颖类中调整特征表示，同时在预先训练的主链中保持语义线索。我们在具有不同骨架的各种几种射击分割方法上评估了{\ em单数值微调（SVF）}方法。我们在Pascal-5 $^i $和Coco-20 $^i $上都获得了最先进的结果。希望这个简单的基准将鼓励研究人员重新考虑骨干微调在几次环境中的作用。源代码和模型将在\ url {https://github.com/syp2ysy/svf}上获得。

基于弱监管的像素 - 明显的密集预测任务当前使用类注意映射（CAM）以产生伪掩模作为地面真理。然而，现有方法通常取决于诱人的训练模块，这可能会引入磨削计算开销和复杂的培训程序。在这项工作中，提出了语义结构知识推断（SSA）来探索隐藏在基于CNN的网络的不同阶段的语义结构信息，以在模型推断中产生高质量凸轮。具体地，首先提出语义结构建模模块（SSM）来生成类别不可知语义相关表示，其中每个项目表示一个类别对象和所有其他类别之间的亲和程度。然后，探索结构化特征表示通过点产品操作来抛光不成熟的凸轮。最后，来自不同骨架级的抛光凸轮融合为输出。所提出的方法具有没有参数的优点，不需要培训。因此，它可以应用于广泛的弱监管像素 - 明智的密集预测任务。对弱势监督对象本地化和弱监督语义分割任务的实验结果证明了该方法的效力，这使得新的最先进的结果实现了这两项任务。

作为人类，我们通过我们所有的感官来驾驭世界，使用每个人从每个人纠正其他人。我们介绍了Merlot Reserve，一个模型，该模型是联合随着时间的推移而表示视频的模型 - 通过从音频，字幕和视频帧学习的新培训目标。给出了一个视频，我们用掩模令牌替换文本和音频的片段;该模型通过选择正确的蒙版片段来学习。我们的目标比替代方面更快地学习，并在规模上表现良好：我们预先逼近2000万YouTube视频。经验结果表明，Merlot Reserve学会通过所有组成模式的视频的强烈陈述。在FineTuned时，它在VCR和TVQA上为VCR和TVQA进行了新的最先进，优先于前勤工作分别为5％和7％。消融表明，两个任务都受益于音频预制 - 甚至录像机，围绕图像中心的QA任务（没有声音）。此外，我们的客观使开箱即用的预测，揭示了强大的多式联合致辞理解。在一个完全零拍摄的环境中，我们的模型在四个视频理解任务中获得竞争结果，甚至优于最近提出的定位推理（星）基准的监督方法。我们分析为什么包含音频导致更好的视觉语言表示，这表明未来研究的重要机会。我们通过讨论多式联运预测的道德和社会影响来得出结论。

可以代表和描述环境声音的机器具有实际潜力，例如，用于音频标记和标题系统。普遍的学习范式已经依赖于并行音频文本数据，但是，Web上几乎没有可用。我们提出了vip-ant，它在不使用任何并行音频文本数据的情况下诱导\ textbf {a} udio- \ textBF {t} EXT对齐。我们的主要思想是在双模形图像文本表示和双模态图像 - 音频表示之间共享图像模型;图像模态用作枢轴，并将音频和文本连接在三模态嵌入空间中。在没有配对的音频文本数据的困难零拍设置中，我们的模型在ESC50和US8K音频分类任务上展示了最先进的零点性能，甚至超过了披肩标题的领域的监督状态检索（带音频查询）2.2 \％R @ 1。我们进一步调查了最小音频监控的情况，发现，例如，只有几百个监督的音频文本对将零拍音频分类精度提高8 \％US8K。然而，为了匹配人类奇偶校验，我们的经验缩放实验表明我们需要大约2米$ 2 ^ {21} \约2M $监督的音频标题对。我们的工作开辟了新的途径，用于学习音频文本连接，几乎没有并行音频文本数据。

In this paper, we propose a robust 3D detector, named Cross Modal Transformer (CMT), for end-to-end 3D multi-modal detection. Without explicit view transformation, CMT takes the image and point clouds tokens as inputs and directly outputs accurate 3D bounding boxes. The spatial alignment of multi-modal tokens is performed implicitly, by encoding the 3D points into multi-modal features. The core design of CMT is quite simple while its performance is impressive. CMT obtains 73.0% NDS on nuScenes benchmark. Moreover, CMT has a strong robustness even if the LiDAR is missing. Code will be released at https://github.com/junjie18/CMT.

Given the increasingly intricate forms of partial differential equations (PDEs) in physics and related fields, computationally solving PDEs without analytic solutions inevitably suffers from the trade-off between accuracy and efficiency. Recent advances in neural operators, a kind of mesh-independent neural-network-based PDE solvers, have suggested the dawn of overcoming this challenge. In this emerging direction, Koopman neural operator (KNO) is a representative demonstration and outperforms other state-of-the-art alternatives in terms of accuracy and efficiency. Here we present KoopmanLab, a self-contained and user-friendly PyTorch module of the Koopman neural operator family for solving partial differential equations. Beyond the original version of KNO, we develop multiple new variants of KNO based on different neural network architectures to improve the general applicability of our module. These variants are validated by mesh-independent and long-term prediction experiments implemented on representative PDEs (e.g., the Navier-Stokes equation and the Bateman-Burgers equation) and ERA5 (i.e., one of the largest high-resolution data sets of global-scale climate fields). These demonstrations suggest the potential of KoopmanLab to be considered in diverse applications of partial differential equations.

Rankings are widely collected in various real-life scenarios, leading to the leakage of personal information such as users' preferences on videos or news. To protect rankings, existing works mainly develop privacy protection on a single ranking within a set of ranking or pairwise comparisons of a ranking under the $\epsilon$-differential privacy. This paper proposes a novel notion called $\epsilon$-ranking differential privacy for protecting ranks. We establish the connection between the Mallows model (Mallows, 1957) and the proposed $\epsilon$-ranking differential privacy. This allows us to develop a multistage ranking algorithm to generate synthetic rankings while satisfying the developed $\epsilon$-ranking differential privacy. Theoretical results regarding the utility of synthetic rankings in the downstream tasks, including the inference attack and the personalized ranking tasks, are established. For the inference attack, we quantify how $\epsilon$ affects the estimation of the true ranking based on synthetic rankings. For the personalized ranking task, we consider varying privacy preferences among users and quantify how their privacy preferences affect the consistency in estimating the optimal ranking function. Extensive numerical experiments are carried out to verify the theoretical results and demonstrate the effectiveness of the proposed synthetic ranking algorithm.

Due to their ability to offer more comprehensive information than data from a single view, multi-view (multi-source, multi-modal, multi-perspective, etc.) data are being used more frequently in remote sensing tasks. However, as the number of views grows, the issue of data quality becomes more apparent, limiting the potential benefits of multi-view data. Although recent deep neural network (DNN) based models can learn the weight of data adaptively, a lack of research on explicitly quantifying the data quality of each view when fusing them renders these models inexplicable, performing unsatisfactorily and inflexible in downstream remote sensing tasks. To fill this gap, in this paper, evidential deep learning is introduced to the task of aerial-ground dual-view remote sensing scene classification to model the credibility of each view. Specifically, the theory of evidence is used to calculate an uncertainty value which describes the decision-making risk of each view. Based on this uncertainty, a novel decision-level fusion strategy is proposed to ensure that the view with lower risk obtains more weight, making the classification more credible. On two well-known, publicly available datasets of aerial-ground dual-view remote sensing images, the proposed approach achieves state-of-the-art results, demonstrating its effectiveness. The code and datasets of this article are available at the following address: https://github.com/gaopiaoliang/Evidential.

A noisy training set usually leads to the degradation of the generalization and robustness of neural networks. In this paper, we propose a novel theoretically guaranteed clean sample selection framework for learning with noisy labels. Specifically, we first present a Scalable Penalized Regression (SPR) method, to model the linear relation between network features and one-hot labels. In SPR, the clean data are identified by the zero mean-shift parameters solved in the regression model. We theoretically show that SPR can recover clean data under some conditions. Under general scenarios, the conditions may be no longer satisfied; and some noisy data are falsely selected as clean data. To solve this problem, we propose a data-adaptive method for Scalable Penalized Regression with Knockoff filters (Knockoffs-SPR), which is provable to control the False-Selection-Rate (FSR) in the selected clean data. To improve the efficiency, we further present a split algorithm that divides the whole training set into small pieces that can be solved in parallel to make the framework scalable to large datasets. While Knockoffs-SPR can be regarded as a sample selection module for a standard supervised training pipeline, we further combine it with a semi-supervised algorithm to exploit the support of noisy data as unlabeled data. Experimental results on several benchmark datasets and real-world noisy datasets show the effectiveness of our framework and validate the theoretical results of Knockoffs-SPR. Our code and pre-trained models will be released.